Image processing device, image processing method and permanent computer-readable medium

ABSTRACT

A storage portion stores ( 220 ) an image data and a distance data including a respective subject and a lens focus. A display ( 180 ) displays a display image based on the image data, and an operation regarding a depth of field and a focus position being performed. An image adjusting portion ( 320 ) detects a first operation and a second operation performed referring to the display ( 180 ), and adjusts a focus distance according to the first operation and the distance data as well as adjusts a depth of field according to the second operation and the distance data for the image data to generate the display image.

TECHNICAL FIELD

The present invention relates to an image processing device, an imageprocessing method, and a non-transitory computer readable media.

BACKGROUND ART

It is known that in a general single-lens reflex camera a depth of fieldcan be varied by changing a stop of lens. However, in a small imagepickup device such as a cellular phone and the like there is a case thatthe stop cannot be equipped due to a problem of cost or size.

Then, in recent years a technique which is called as a light field(light irradiating field) camera is gained attention. In the case ofusing the light field camera, a depth of field of image data can bechanged after imaging. For example, the technique concerning the lightfield camera is described in patent literature 1 to patent literature 3and the like. The light field camera is the technique of reconstructingthe image data which is acquired by using a plenoptic camera to generatea refocused image data

CITATION LIST Patent Literature

-   patent literature 1: Japanese Unexamined Patent Application    Publication No. 2012-191376-   patent literature 2: Japanese Unexamined Patent Application    Publication No. 2012-147167-   patent literature 3: Japanese Unexamined Patent Application    Publication No. 2012-142918

Non Patent Literature

-   non patent literature 1: Nikkei-electronics 20 Aug. 2012, “arayuru    tokoroni kasikoi Camera”

SUMMARY OF INVENTION Technical Problem

In the field of the afore-mentioned light field camera, the technique tochange the depth of field after imaging is generally used. However, itis not sufficiently considered by which kind of interface is used to setthe depth of field or focus in changing the depth of field on the userside. Accordingly, the problem occurred that the image data in which thedepth of field or focus is adequately set can not be rapidly obtained.

The present invention has been made to solve the above-mentionedproblems, and has a main object to provide an image processing device,an image processing method, and a non-transitory computer readablemedia, which have an operation interface to adjust and operate a depthof field in addition to adjust a focus.

Solution to Problem

An aspect of an image processing device in accordance with the presentinvention is an image processing device comprising:

storage means for storing an image data including a distance databetween a respective subject and a lens focus;

display means for displaying a display image based on the image data,and an operation regarding a depth of field and a focus position beingperformed; and

image adjusting means for detecting a first operation and a secondoperation performed referring to the display means, and adjusting afocus distance according to the first operation and the distance data aswell as adjusting a depth of field according to the second operation andthe distance data for the image data to generate the display image.

An aspect of an image processing method in accordance with the presentinvention is an image processing method comprising:

a display step of displaying a display image based on an image dataincluding a distance data between a respective subject and a lens, andan operation regarding a depth of field and a focus position beingperformed; and

an image control step of detecting a first operation and a secondoperation performed referring to the display image, and adjusting afocus distance according to the first operation and the distance data aswell as adjusting a depth of field according to the second operation andthe distance data for the image data to generate the display image.

An aspect of an image processing method in accordance with the presentinvention is a non-transitory computer readable media for causing acomputer to execute:

a step of reading out an image data including a distance data between arespective subject and a lens from a storage device; and an imagecontrol step of detecting a first operation and a second operationperformed referring to the display of the display image based on theimage data, and adjusting a focus distance according to the firstoperation and the distance data as well as adjusting a depth of fieldaccording to the second operation and the distance data for the imagedata to generate the display image.

Advantageous Effects of Invention

According to the present invention, it is possible to provide an imageprocessing device, an image processing method, and a non-transitorycomputer readable media, which have an operation interface to adjust andoperate a depth of field in addition to adjust a focus.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing a configuration of an image processingdevice 1 according to an exemplary embodiment 1;

FIG. 2 is a block diagram schematically showing a control structure ofan image processing device 1 according to an exemplary embodiment 1;

FIG. 3 is a concept diagram showing an operation example of an imageprocessing device 1 according to an exemplary embodiment 1;

FIG. 4 is a diagram showing slide bar S1 displayed on display 180according to an exemplary embodiment 1;

FIG. 5 is a diagram showing slide bar S1 displayed on display 180according to an exemplary embodiment 1;

FIG. 6 is a concept diagram showing an operation example of an imageprocessing device 1 according to an exemplary embodiment 1;

FIG. 7 is a concept diagram showing an operation example of an imageprocessing device 1 according to an exemplary embodiment 1;

FIG. 8 is a concept diagram showing an operation example of an imageprocessing device 1 according to an exemplary embodiment 1;

FIG. 9A is a concept diagram showing a focus position set according toan exemplary embodiment 1;

FIG. 9B is a concept diagram showing a focus position set according toan exemplary embodiment 1;

FIG. 10 is a concept diagram showing an operation example of an imageprocessing device 1 according to an exemplary embodiment 1;

FIG. 11 is a diagram showing slide bar S1 displayed on display 180according to an exemplary embodiment 1; and

FIG. 12 is a block diagram showing a configuration of an imageprocessing device 1 according the present invention.

DESCRIPTION OF EMBODIMENTS Example 1

Exemplary embodiments of the present invention will be described belowwith reference to the drawings. FIG. 1 is a block diagram showing aconfiguration of an image processing device according to an exemplaryembodiment 1. Note that hereinafter image processing device is explainedas a smart phone, however it is not necessarily limited to this. Imageprocessing device 1 may be an optional terminal having an operationinterface (operation portion) such as a touch panel and the like.

The image processing device 1 includes an image portion 100, RAM (RandomAccess memory) 110, an image processing circuit 120, a YC convertingcircuit 130, a compressing and extending circuit 140, a mediancontroller 150, a memory card 160, a LCD driver 170, a display 180, aflash memory 190, CPU (Central Processing Unit) 200, and data bus 210.

The image portion 100 includes a lens 101, a micro array lens 102, a CCD(Charged Coupled Device) 103, and a data processing portion 104. Thelens 101 is a main lens for imaging a subject. The micro array lens 102is for example a plurality of micro lens which are two-dimensionally ina matrix disposed, and is disposed on an image formation face of thelens 101. Each of the micro lens whose plane shape, is a circle shape ora hexagon shape is constituted of a solid lens, a liquid crystal lens, aliquid lens, a diffraction lens and the like.

In the CCD 103 a plurality of light receiving elements for accumulatingan electric charge according to light receiving amount entered as alight from the micro array lens 102 via its light receiving face isdisposed. The CCD 103 outputs an electric charge which is accumulated inthe respective light receiving element by imaging as an image signal.The micro array lens 102 is constituted of a plurality of micro lens.Accordingly, a physical distance between respective subject and lensfocus (for example in case of imaging a child the physical distancebetween the position of the lens focus of the image portion 101 and theposition of the child (10 m)) can be obtained. Note that a CMOS(Complementary Metal-Oxide Semiconductor) may be used instead of the CCD103. The data processing portion 104 is constituted of for example a CDS(Correlated Double Sampling), an amplifier, A/D (Analog/Digital)converter and the like, performs a general process in a general imagedevice, and generates an image data including a distance between therespective subject within image range and the lens focus to store in RAM110.

Note that the afore-mentioned constitution of the image portion 100 isan only exemplification. The constitution may be optional if image datawhich includes a distance between the respective subject within imagerange and the lens focus can be obtained. The constitution may be knownlens module for realizing the structure of the light field camera. Forexample, the image portion 100 may include a lens for a right eye and alens for a left eye, and may generate a parallax image data by therespective lens. Regarding the process of obtaining the distance data ofthe subject by the light field camera and the like, the structure issame as the structure described in non patent literature 1 and the like,accordingly, properly refer to it.

The image processing circuit 120 executes the process such as a grayscale conversion, a white balance correction, a gamma correction and thelike. YC converting circuit 130 converts a data after imaging to abrightness signal Y and a color difference signal Cr, Cb. Thecompressing and extending circuit 140 compresses image data at anoptional timing according to a predetermined compression format to writein an optional storage portion (for example memory card 160, flashmemory 190).

The median controller 150 writes and read all sorts of data for thememory card 160. The memory card 160 is a storage portion in which datais read and written via the median controller 150. The memory card 160is constituted to be capable of being attached and detached to the imageprocessing device 1. The display 180 is driven by LCD driver 170. TheLCD driver 170 converts image data which is read out from all sorts ofstorage portion (for example the memory card 160, the flash memory 190and the like) and is performed digital processing by the CPU 200 to forexample an analog signal in NSC (National Television System Committee)format, and outputs the converted an analog signal to the display 180.

The display 180 is provided on the front face of the image processingdevice 1 (smart phone). The display 180 is realized by a crystal displaydevice, an organic EL (Electro Luminescence) display and the like,however it is not necessarily limited to this. The display 180corresponds to a multi-touch such as a capacitance type, a resistancefilm type, an electromagnetic type and the like. The display 180functions in an image mode as an electric view finder to display theimage in real time. Further, the display 180 functions in a playbackmode as a monitor to play back and display the image stored in all sortsstorage (for example the memory card 160, the flash memory 190 and thelike).

The display includes a touch sensor 181. The touch sensor 181 detects acontact position when a user's hand or finger contacts a display screenof the display 180. Then, the touch sensor 181 detects operations suchas the so called pinch-in (an operation narrowing distance between twofingers), pinch-out (an operation widening distance between two fingers)and the like.

The flash memory 190 is a storage portion that stores all sorts of data(including image data). Note that the image processing device 1 properlyincludes a HDD (a hard disk drive) not shown as a storage portion.

CPU (Central Processing Unit) 200 is electrically connected to therespective processing portion in the image processing device 1 via thedata bus 210. Data is inputted into the CPU 200 from all sorts ofswitches (for example a source switch) on the not-shown image processingdevice 1 (a smart phone in this example) and the afore-mentioned touchsensor 181. The CPU 200 reads out a program from all sorts of storagedevices (the not-shown HDD, memory card 160, flash memory 190 and thelike) and controls the respective processing portion as well as performsthe process of generating a display image displayed on the display 180from the image data and distance data generated by the image portion.The generating process of the display image will be in detail explainedreferring to FIG. 2.

The control structure shown in FIG. 2 (an image data read-out portion300, an sensor information input portion 310, an image adjusting portion320, an adjusting interface control portion 330) is realized byexecuting a program by CPU 200.

The program can be stored and provided to a computer using any type ofnon-transitory computer readable media. Non-transitory computer readablemedia include any type of tangible storage media. Examples ofnon-transitory computer readable media include magnetic storage media(such as floppy disks, magnetic tapes, hard disk drives, etc.), opticalmagnetic storage media (e.g. magneto-optical disks), CD-ROM (compactdisc read only memory), CD-R (compact disc recordable), CD-R/W (compactdisc rewritable), and semiconductor memories (such as mask ROM, PROM(programmable ROM), EPROM (erasable PROM), flash ROM, RAM (random accessmemory), etc.). The program may be provided to a computer using any typeof transitory computer readable media. Examples of transitory computerreadable media include electric signals, optical signals, andelectromagnetic waves. Transitory computer readable media can providethe program to a computer via a wired communication line (e.g. electricwires, and optical fibers) or a wireless communication line.

The image data read-out portion 300 successively reads out the imagedata taken by the image portion 100 and stored in the respective storageportion (the memory card 160, flash memory 190, the not-shown hard diskand the like) and the distance data related with the image data. Then,the image data read-out portion 300 successively reads out the imagedata and the distance data in case that the image portion 100 takes amoving picture. The image data read-out portion 300 provides theread-out image data and distance data for the image adjusting portion320 and the adjusting interface control portion 330.

The movement of the finger on the display 180 detected by the touchsensor 181 is inputted into the sensor information input portion 310. Indetail, a direction of the display 180 (a direction of the display 180acquired by the not-shown acceleration sensor), a position of a fingeror a contact state of fingers (how many fingers contacts on the display180), a movement distance of a finger, a movement direction are inputtedinto the sensor information input portion 310. The sensor informationinput portion 310 provides the acquired information of the movement offinger on the display 180 (the position of finger, the contact number offingers, the movement distance of finger, the direction of the display180) for the image adjusting portion 320 and the adjusting interfacecontrol portion 330.

The image adjusting portion 320 detects whether a first operation and asecond operation is performed based on the movement distance of finger,the position of finger, and the movement direction of finger. The firstoperation is an operation to adjust a focus, and an example of theoperation is later described referring to FIG. 3 and the like. Thesecond operation is an operation to adjust a depth of field and anexample of the operation is later described referring to FIG. 3 and thelike.

The image adjusting portion 320 adjusts the focus of the image data readout by the image data read-out portion 300 according to the detection ofthe first operation to adjust the focus. In the adjustment the distancedata read out by the image data read-out portion 300 is used. Then, theimage adjusting portion 320 may adjust a focal distance (adjust a focalposition) performed in a technique field of a general light fieldcamera.

Furthermore, the image adjusting portion 320 adjusts the depth of fieldof the image data read out by the image data read-out portion 300according to the detection of the second operation to adjust the depthof field. In the adjustment the distance data read out by the image dataread-out portion 300 is used. Then, the image adjusting portion 320 mayadjust a focal distance performed in a technique field of a generallight field camera.

The adjusting interface control portion 330 detects whether a firstoperation and a second operation is performed based on the movementdistance of finger, the position of finger, and the movement directionof finger. The adjusting interface control portion 330 controls thedisplay of an adjusting interface (preferably a slide bar shown in FIG.3) displayed on the display 180 according to the detection. Asafter-mentioned, the slide bar is displayed with overlapping on thedisplay image.

Next, the respective operation on the display 180 according to theexemplary embodiment. FIG. 3 is a concept diagram showing a operationexample of the display 180. Nora that in the below explanation it isassumed that the display is operated by using a thumb and a forefingerof user's right hand, however any of ten fingers of both hands may beused.

The slide bar S1 is displayed on the display 180. The height length ofthe slide bar S1 indicates all focus range of the taken image data. Anupper direction of the slide bar S1 indicates a distant side (Far) and alower direction indicates this side (Near). In a knob S2 of the slidebar S1 an upper side indicates a focusing end of the distant side and alower side indicates a focusing end of the near side.

In case user selects a reference mode (a mode for reading a taken image)of the image processing device, a live view scene appears on the display180. The image processing device 1 have the afore-mentioned function ofthe light field camera and acquires the image data and the distance dataof the image object and stores them in the storage device.

The adjusting interface control portion 330 displays the slide bar S1 onthe display 180 with overlapping on the display image when the touchsensor 181 detects the contact of user's finger. As shown in the figurethe slide bar S1 has a vertically long shape. Accordingly, in case thetouch sensor 181 detects that user touches the right half of the screen,the adjusting interface control portion 330 displays the slide bar S1 onthe right end of the display 180. On the other hand, in case the touchsensor 181 detects that user touches the left half of the screen, theadjusting interface control portion 330 displays the slide bar S1 on theleft end of the display 180. In case the touch sensor 181 detects thatuser touches the approximate center portion of the screen in ahorizontal direction, the adjusting interface control portion 330displays the slide bar S1 on the right end of the display 180considering there are many user with a right hand dominance. Note thatthe display position of the slide bar S1 may be displayed in the fixedposition according to the settings set by user.

In case user widens a space between finger F1 and F2 in y axis direction(the so-called pinch-out), the image adjusting portion 320 and theadjusting interface control 330 detects the extension degree (themovement distance of the two fingers in y axis direction) according toinput from the touch sensor 181. In the example of FIG. 3 the pinch-out(or the pinch-in) in y axis direction becomes the second operation toadjust the depth of field. The adjusting interface control portion 330widens the width of the knob S2 according to the detected extensiondegree. The image adjusting portion 320 widens the focusing end of thedistant side and the focusing end of the near side and changes the depthof field.

On the other hand, in case user narrows the space between finger F1 andF2 in y axis direction (the so-called pinch-in), the touch sensor 181detects the reduction degree. The adjusting interface control portion330 narrows the width of the knob S2 according to the detected reductiondegree. The image adjusting portion 320 narrows the focusing end of thedistant side and the focusing end of the near side and changes the depthof field. This is effective in case of wanting to display a gradatingbackground with an only subject being focused.

In case the space of the user's fingers F1 and F2 is kept and fingers F1and F2 slides in y axis direction, the touch sensor 181 detects themovement direction and the movement distance. In the example of FIG. 3,an operation to keep the space between two fingers F1 and F2 in y axisdirection and move them is detected. The touch sensor 181 detects themovement degree. The adjusting interface control portion 330 moves theposition of the knob S2 according to the detected movement degree. Atthis time, the width of the knob S2 (the length from the upper end tothe lower end) is not changed. Further, the image adjusting portionadjusts the focus position by simultaneously adjusting the focus end ofthe distant side and the focus end of the near side according to thedetected movement degree. That is, a manual focus operation is realized.

Note that the pinch-in (or the pinch-out) in x axis direction may beassigned to a general zoom process. Accordingly, a general zoom processcan be realized and the depth of field and the focus position can beadjusted. Then the axis direction is an example. The pinch-in (thepinch-out) in x axis direction may be assigned to the adjustment of thedepth of field (the second operation) (that is, the slide bar S1 may beextended and provided in x axis direction).

Note that the image adjusting portion 320 may adjust the so-called blur(Bokeh) according to the setting width of the depth of field. Thespecific example is explained referring to FIG. 4 and FIG. 5. FIG. 4 andFIG. 5 are a diagram showing slide bar S1 displayed on display 180 indisplaying the same display image. The depth of field in the slide barS1 shown in FIG. 4 is set to be narrower compared with the depth offield in the slide bar S1 shown in FIG. 5 That is, the width of area tobe focused becomes small. With the setting width of the depth of fieldbeing smaller, the image adjusting portion 320 performs the imageprocess such that the blur of the subject imaged outside the depth offield is stronger. For example, the blur of the subject imaged within Arange of FIG. 4 is given more strongly than the blur of the subjectimaged within A range of FIG. 5. Note that in the image process to givethe blur a manner in a general digital image process may be used.

In a general single-lens reflex camera in the area outside the depth offield with being farther from the depth of field, the blur is stronger.Here, the strength of the blur in the single-lens reflex camera dependson a lens feature of the camera. For example, in the single-lens reflexcamera Z the strength of the blur far from the area indicating the depthof field by 10 m is x. However, in the single-lens reflex camera W thestrength of the blur far from the area indicating the depth of field by10 m is 2x. Thus, they are different.

Therefore, the image adjusting portion 320 may have a plurality oftables regarding a manner to give the blur, and may adjust the strengthof the blur by substituting the distance from the area of the depth offield for the table selected by user. The table is a table related withthe distance from the area of the depth of field and the strength of theblur. The plurality of tables included in the image adjusting portion320 may be a table corresponding to the lens of the existing respectivesingle-lens reflex camera. Accordingly, even if an image data taken bythe image portion 100 without a stop is displayed on the display 180, adisplay effect can be given in the same manner as a single-lens reflexcamera which user usually uses.

Next, referring to FIG. 6, it is a diagram showing a second operationexample of the image processing device 1 according to the exemplaryembodiment. In the second operation example, the rotation operation offingers in a multi-touching state is assigned to the second operation(the adjustment operation of the depth of field).

In case user rotates fingers F1 and F2 (moving fingers F1 and F2 inarrow direction of FIG. 6), the touch sensor 181 detects the movementdirection and movement distance. In case the fingers rotatesclockwisely, the image adjusting portion 320 performs the imageadjustment so that the depth of field becomes wider according to themovement distance. Meanwhile, in case the fingers rotatescounterclockwisely, the image adjusting portion 320 performs the imageadjustment so that the depth of field becomes narrower according to themovement distance. Further, the adjusting interface control portion 330changes the size of the knob S2 according to the rotation width offingers F1 and F2 by user. The rotation direction is a example. In caseof rotating fingers counterclockwisely, it may be controlled so that thedepth of field becomes wider.

As a third operation example, the example to set the depth of field andthe focus position by using only one finger will be explained. A userlongly pushes the subject to be focused in the image display with onefinger F1. In the example of FIG. 7, the user longly pushes a face of agirl which is the subject. Then, the physical distance between the faceof the girl and the lens focus is acquired by the afore-mentioned imageportion 100 and stored in all sorts of storage portions. The imageadjusting portion 320 and the adjusting interface control portion 330detects this long push as the afore-mentioned first operation. Then, theimage adjusting device 320 performs the image adjustment so that thisposition of the face of the girl is the focus position. Similarly, theadjusting interface control portion 330 adjusts the position of the knobS2 on the slide bar S1 according to the position of the face of thisgirl (the distance from lens).

Further, the image adjusting portion 320 and the adjusting interfacecontrol portion 330 detects the second operation according to theoperation of one finger. For example, as shown in FIG. 7, the imageadjusting portion 320 and the adjusting interface control portion 330detects the operation drawing a circle as the second operation. In thiscase, the image adjusting portion 320 and the adjusting interfacecontrol portion 330 may adjust by widening the depth of field indetecting the clockwise operation and by narrowing the depth of field indetecting the counterclockwise operation.

Similarly, the depth of field and the focus position can also be set byusing a pen device. In case of using the pen device, the imageprocessing device 1 may detect the first operation (the operation oflongly pushing on one point) and the second operation (the operation ofdrawing a circle) explained as the third operation example. The imageprocessing device 1 can also detect the first operation and the secondoperation considering the push state of the button and the like which isprovided with a pen-tablet.

Note that in case of using the pen-device, the image adjusting portion320 may set the depth of field considering a thickness and the like ofthe pen point.

Next, referring to FIG. 8, a fourth setting example according to thedepth of field and the focus position will be explained. To set thefocus position, similarly to FIG. 7 (the third operation example), thelong push is performed. A user successively pushes the subject of object(F1-1, F1-2, and F1-3 of FIG. 7) within a predetermined time interval.The physical distance between the respective subject position and thelens focus is acquired by the image portion 100 and stored in all sortsof storage portions as afore-mentioned. The image adjusting portion 320and the adjusting interface control portion 330 recognizes among aplurality of the subjects (the subjects selected by F1-1 to F1-3) onebetween the shortest distance and the longest distance from lens as thedepth of field. Then, the image adjusting portion 320 performs imageadjustment according to the setting of the recognized depth of field.Simultaneously, the adjusting interface control portion 330 adjusts thewidth of the knob S2 on the slide bar S1 according to the setting of therecognized depth of field.

Next, referring to FIG. 9, the variation of setting methods of the focusposition by the image adjusting portion 320 will be explained. FIG. 9 isa concept diagram showing the physical position and the focusing degreeof the respective position regarding the focus position specified in thelens position of the image processing device 1 (a white circle portionin the figure) and the display 180.

FIG. 9A is a concept diagram showing a first variation of the setting ofthe focus position. The image adjusting portion320 calculates a distancebetween the physical position indicated by the touching position of thedisplay 180 (the black circle portion in the figure) and the physicalposition of the lens of the image processing device 1 (the white circleposition in the figure) from the distance data. Then, the imageadjusting portion 320 performs image adjustment so that all of thepoints (the respective point of a2 in the figure) which is same distanceas the distance are focused. That is, the image adjusting portion 320performs image adjustment so that the focus position is not point butthe respective point where the distance from the lens position is apredetermined distance is focused.

FIG. 9A is a concept diagram showing a first variation of the setting ofthe focus position. The image adjusting portion320 calculates a distancebetween the physical position indicated by the touching position of thedisplay 180 (the black circle portion in the figure) and the physicalposition of the lens of the image processing device 1 (the white circleposition in the figure) from the distance data. Then, the imageadjusting portion 320 performs image adjustment by defining one point ofthe physical position (the black circle in the figure) indicated by thetouching position as a center of the focus position. That is, the imageadjusting portion 320 performs image adjustment assuming the center ofthe focus position is one point.

A user may select and use either the focus adjustment shown in FIG. 9(A)or the focus adjustment shown in FIG. 9(B) from a not-shown settingscene.

Note that in the above description the various operation example (thefirst operation to the fourth operation) is explained. However, thesetting interface relating the operation with the setting contents maybe displayed on the display 180, and a user may freely relate therespective setting contents (a zoom, a depth of field, a focus positionand the like) with operation contents (e.g. the pinch-in (pinch-out) inx axis direction).

Further, the afore-mentioned operation example is an example to the end.The different operation may also be assigned as the first operation andthe second operation. For example, the process of successively touchingone point (a plurality of taps) may be assigned. That is, the differentoperation from a zoom may be assigned to the first operation or thesecond operation.

Next, the effect of the image processing device 1 according to theexemplary embodiment will be explained. The image processing device 1according to the exemplary embodiment detects the first operation toadjust the focus position and the second operation to adjust the depthof field as afore-mentioned with the display image displayed on thedisplay 180, and performs image adjustment with the constitution of thelight field camera according to the operations. Accordingly, a user canadjust the focus distance and the depth of field by the intuitionaloperation while referring to the display image.

As shown in FIG. 3, the operation of performing a pinch-in or apinch-out in a definite direction (y axis direction in FIG. 3) differentfrom direction of the zoom is assigned to the second operation. Thereby,a user can set the depth of field by the operation different from thezoom (e.g. the pinch-in or the pinch-out in x axis direction). Further,the first operation and the second operation regarding the adjustment ofthe depth of field or the focus position are assigned to generalpinch-in, pinch-out, movement of fingers and the like. Thereby, thesetting of the focus position and the depth of field is facilitated formany users.

Further, as shown in FIG. 4, the operation of rotating fingers (movingin approximate circle direction) is assigned to the second operation.Thereby, a user can set the depth of field by the sensibly sameoperation as the stop adjustment.

Further, the image processing device 1 displays the slide bar S1 whichis an adjustment interface on the display 180. Accordingly, the focusposition and depth of field of the display image displayed at presentcan be rapidly grasped. Furthermore, the adjusting interface controlportion 330 adjusts the display position of the slide bar S1 (a rightend, a left end and the like) according to the touching position of auser. Accordingly, the slide bar S1 can be displayed at the positionwhich is not an obstacle for a user.

In addition, the image processing device 1 changes the blur effectaccording to the width of the area of the depth of field as describedreferring to FIG. 4 and FIG. 5. Thereby, the same display effect as ageneral single-lens reflex camera can be realized.

The afore-mentioned exemplary embodiment is only an example regardingapplications of a technique idea obtained by the inventor. That is, thetechnique idea is not limited to the only afore-mentioned exemplaryembodiment, various changes may be performed.

Note that in the afore-mentioned explanation it is described that theimage processing device 1 is a device including the image portion 100(e.g. a smart phone, a digital camera and the like), however it is notlimited to this. For example, the image processing device 1 may be ageneral personal computer. In this case, the image processing device 1may be constituted to be able to download the image data (including thedistance data) via an internet or read out the image data (including thedistance data) from the attachable and detachable storage device (e.g. aUSB (Universal Serial Bus) memory). A user performs the first operationand the second operation with a mouse referring to the display device(e.g. a liquid crystal display device) connected to a computer.

Furthermore, the image processing device 1 may be constituted to combinean auto focus function to focus a plurality of points with theafore-mentioned operations. Further, the display of the adjustinginterface (S1) is necessarily not essential, it may not be displayedaccording to the user setting.

Further, in the setting of the depth of field, the movement distance ofthe focusing end of the near side may be different from the movementdistance of the focusing end of the distant side. Referring to FIG. 10,it will be explained. FIG. 10 is a setting screen of the depth of field.In the example it is explained assuming that the depth of field iswidened by the rotational operation of fingers.

In case of setting so that the depth of field becomes wider (e.g. incase of clockwisely rotating fingers), the image adjusting portion 320and the adjusting interface control portion 330 detects the adjustingdegree (the rotation degree of the fingers). Then, the image adjustingportion 320 and the adjusting interface control portion 330 performadjustment so that the adjustment width (namely movement distance) ofthe focusing end of the near side is different from the adjustment widthof the focusing end of the distant side. Referring to the slide bar S1of FIG. 1, it is explained in detail.

In case of setting so that the depth of field becomes wider, the imageadjusting portion 320 and the adjusting interface control portion 330set the movement distance of the focusing end of the distant side to bewider than the movement distance of the focusing end of the near side.Thereby, as shown in FIG. 10 in case that a girl to be focused as asubject has a flower, the image in which the flower in addition to thegirl to be the subject is clearly displayed can be obtained. Further,the blur of the distant side (namely a background) is stronger, theimage having the novel blur sense and indicating the subject of thedisplay object to be more three-dimensional (clearly) can be obtained.

Note that the image adjusting portion 320 and the adjusting interfacecontrol portion 330 perform image adjustment without changing the centerof the depth of field (the position corresponding to the white circle ofthe slide bar S2) (For example, the girl is positioned at the center ofthe depth of field).

This method to give the blur sense can not be realized by a generalsingle-lens reflex camera, thereby the novel image can be obtained. Notethat the image adjusting portion 320 and the adjusting interface controlportion 330 set the movement distance of the focusing end of the distantside to be narrower than the movement distance of the focusing end ofthe near side. The movement distance of the focusing end may be properlychanged by the user setting.

Finally, referring to FIG. 12, the essential portion of the imageprocessing device 1 according to the present invention will be explainedagain. FIG. 12 is a block diagram showing a summary of an imageprocessing device 1 according the present invention. The storage portion220 is a general term such as the flash memory 190, memory card 160,not-shown harddisk and the like according to FIG. 1, and stores theimage data acquired by the image portion 100 (or an image device outsidethe image processing device 1) having the constitution of the lightfield camera. Further, the storage portion 220 stores the image data aswell as the distance data between the respective subject and the lensfocus.

The display 180 functions as a display portion to display the generateddisplay image based on the image data. The display 180 has for examplethe touch sensor 181 and the first operation and the second operationregarding the depth of field and the focus position are performed.

The CPU 200 is an arithmetic device to control the image processingdevice 1, and reads out a program from the storage portion 200 andexecutes it. The CPU 200 operates as the image adjusting portion 320 byexecuting the program according to the control of the light fieldcamera.

The image adjusting portion 320 detects the first operation and thesecond operation performed on the display 180. Then the image adjustingportion 320 reads out the image data and the distance data from thestorage portion 220, and adjusts the focus position of the image dataaccording to the first operation (e.g. the parallel movement of thefingers in a multi-touching state). Further, the image adjusting portion320 adjusts the depth of field of the image data according to the secondoperation (e.g. the pinch-in or pinch-out in the multi-touching state).Accordingly, the image adjusting portion 320 generates the display imagedisplayed on the display 180.

This constitution enables obtaining the display image in which the focusposition or the depth of field is simply adjusted according to theoperation on the display portion (the display 180).

While the invention has been particularly shown and described withreference to exemplary embodiments thereof, the invention is not limitedto the afore-mention. It will be understood by those of ordinary skillin the art that various changes in form and details may be made thereinwithout departing from the spirit and scope of the present invention asdefined by the claims.

This application is based upon and claims the benefit of priority fromJapanese patent application No. 2013-031805, filed on Feb. 21, 2013, thedisclosure of which is incorporated herein in its entirety by reference.

REFERENCE SIGNS LIST

-   1 IMAGE PROCESSING DEVICE-   100 IMAGE PORTION-   110 RAM-   120 IMAGE PROCESSING CIRCUIT-   130 YC CONVERTING CIRCUIT-   140 COMPRESSING AND EXTENDING CIRCUIT-   150 MEDIAN CONTROLLER-   160 MEMORY CARD-   170 LCD DRIVER-   180 DISPLAY-   181 TOUCH SENSOR-   190 FLASH MEMORY-   200 CPU-   210 DATA BUS-   220 STORAGE PORTION-   300 IMAGE DATA READ-OUT PORTION-   310 SENSOR INFORMATION INPUT PORTION-   320 IMAGE ADJUSTING PORTION-   330 ADJUSTING INTERFACE CONTROL PORTION

What is claimed is:
 1. A image processing device comprising: a storage portion for storing an image data including a distance data between a respective subject and a lens focus; a display for displaying a display image based on the image data, and an operation regarding a depth of field and a focus position being performed; and an image adjusting portion for detecting a first operation and a second operation performed referring to the display, and adjusting a focus distance according to the first operation and the distance data as well as adjusting a depth of field according to the second operation and the distance data for the image data to generate the display image.
 2. The image processing device according to claim 1, wherein the display includes a touch sensor that detects a contact of a user's finger or all sorts of devices, and is a touch display that displays the display image.
 3. The image processing device according to claim 2, wherein the first operation is an operation of moving user's fingers in a predetermined direction without keeping apart from the touch display while the space between user's two fingers are kept to be approximately same.
 4. The image processing device according to claim 2, wherein the second operation is an operation of widening or narrowing the space between user's two fingers in a predetermined direction while user's fingers are contact with the touch display.
 5. The image processing device according to claim 2, wherein the second operation is an operation of moving in an approximate circle direction while user's one finger or two fingers is contact with the touch display.
 6. The image processing device according to claim 2, wherein the first operation is an operation of pushing the subject for not less than a predetermined time in displaying the display image.
 7. The image processing device according to claim 2, wherein the second operation is an operation of touching the touch display several times within a predetermined time interval in displaying the display image.
 8. The image processing device according to claim 2, further comprising an adjusting display control portion for displaying an adjusting interface indicating a setting degree of the focus position and the depth of field of the display image on the touch display based on the first operation and the second operation performed for the touch display.
 9. The image processing device according to claim 8, wherein the adjusting display control portion changes the display position of the adjusting interface on the touch display according to the contact position of user's fingers on the touch display detected by the touch sensor.
 10. The image processing device according to claim 9, wherein the adjusting display control portion displays the display position of the adjusting interface on the touch display from the center to the right area of the touch display in case that the contact position of user's fingers on the touch display detected by the touch sensor is a predetermined center area.
 11. The image processing device according to claim 1, wherein the image adjusting portion adjusts for the image data the depth of field according to the second operation and the distance data as well as gives a blur effect based on a size of an area of the depth of field and a physical distance from the area, to the subject outside the area of the depth of field.
 12. The image processing device according to claim 1, wherein on adjusting for the image data the depth of field according to the second operation and the distance data, the image adjusting portion adjusts it so that the movement distance of the focusing end of the near side is different from the movement distance of the focusing end of the distant side.
 13. The image processing device according to claim 12, wherein on adjusting for the image data the depth of field according to the second operation and the distance data, the image adjusting portion adjusts it so that the movement distance of the focusing end of the near side becomes narrower than the movement distance of the focusing end of the distant side.
 14. A image processing method comprising: a display step of displaying a display image based on an image data including a distance data between a respective subject and a lens, and an operation regarding a depth of field and a focus position being performed; and an image control step of detecting a first operation and a second operation performed referring to the display image, and adjusting a focus distance according to the first operation and the distance data as well as adjusting a depth of field according to the second operation and the distance data for the image data to generate the display image.
 15. A non-transitory computer readable media for causing a computer to execute: a step of reading out an image data including a distance data between a respective subject and a lens from a storage device; and an image control step of detecting a first operation and a second operation performed referring to the display of the display image based on the image data, and adjusting a focus distance according to the first operation and the distance data as well as adjusting a depth of field according to the second operation and the distance data for the image data to generate the display image. 